A Bayesian Cognitive Approach to Quantifying Software Exploitability Based on Reachability Testing (Extended Version)

Computer hackers or their malware surrogates constantly look for software vulnerabilities in the cyberspace to perform various online crimes, such as identity theft, cyber espionage, and denial of service attacks. It is thus crucial to assess accurately the likelihood that a software can be exploited before it is put into practical use. In this work, we propose a cognitive framework that uses Bayesian reasoning as its first principle to quantify software exploitability. Using the Bayes’ rule, our framework combines in an organic manner the evaluator’s prior beliefs with her empirical observations from software tests that check if the security-critical components of a software are reachable from its attack surface. We rigorously analyze this framework as a system of nonlinear equations, and henceforth perform extensive numerical simulations to gain insights into issues such as convergence of parameter estimation and the effects of the evaluator’s cognitive characteristics.